Intervertebral disc modeling

ABSTRACT

A method is disclosed for spinal anatomy segmentation. In one example, the method includes combining a fully convolutional network with a residual neural network. The method also includes training the combined fully convolutional network with the residual neural network from end to end. The method also includes receiving at least one medical image of a spinal anatomy. The method also includes applying the fully convolutional network with the residual neural network to at least one medical image and segmenting at least one vertebral body from the at least one medical image of the spinal anatomy.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/852,516, filed Apr. 19, 2020, which is a continuation ofInternational Application PCT/US18/56797, filed on Oct. 19, 2018,currently pending, which claims priority to U.S. Provisional ApplicationNo. 62/575,260, filed on Oct. 20, 2017, now expired, the entiredisclosures of which are incorporated herein by reference.

BACKGROUND

This disclosure relates to a method of modeling intervertebral discs.Accurate segmentation of vertebrae from medical images is an importantpreliminary task in image-guided spine surgery. Describing the detailedshape of the vertebrae can considerably help with the early diagnosis,surgical planning, and follow-up assessment of a number of spinaldisorders. In recent years, magnetic resonance imaging (MRI) has becomea valuable, non-invasive tool for vertebral analysis. Unlike computedtomography (CT), MRI does not pose risks associated with exposure toradiation. However, magnetic resonance image analysis encounters severalchallenges for segmentation, such as variability in resolutions andintensity ranges across different scans.

SUMMARY

The needs above, as well as others, are addressed by embodiments ofmethods for modeling intervertebral discs as described in thisdisclosure.

A method is disclosed for spinal anatomy segmentation. The methodincludes combining a fully convolutional network with a residual neuralnetwork. The method also includes training the combined fullyconvolutional network with the residual neural network from end to end.The method also includes receiving at least one medical image of aspinal anatomy. The method also includes applying the fullyconvolutional network with the residual neural network to the at leastone medical image and segmenting at least one vertebral body from the atleast one medical image of the spinal anatomy.

Another method is disclosed for spinal anatomy segmentation. The methodincludes combining a fully convolutional network with a residual neuralnetwork. The method also includes training the combined fullyconvolutional network with the residual neural network from end to end.The method also includes receiving at least one magnetic resonance imageof a spinal anatomy. The method also includes applying the fullyconvolutional network with the residual neural network to the receivedat least one magnetic resonance image and segmenting at least onevertebral body from the at least one magnetic resonance image of thespinal anatomy. The method also includes extracting from the segmentedat least one magnetic resonance image (i) a first three-dimensionalvertebral surface that corresponds to a top vertebra and (ii) a secondthree-dimensional vertebral surface that corresponds to a lowervertebra, wherein the lower vertebra is adjacent to the top vertebra.The method also includes modeling an intervertebral disc mesh based on asurface reconstruction that includes the first three-dimensionalvertebral surface and the second three-dimensional vertebral surface.

DESCRIPTION OF THE FIGURES

FIG. 1 depicts an example method for intervertebral disc modeling,according to an embodiment of the present disclosure.

FIG. 2 depicts an example architecture of a residual neural network,according to an embodiment of the present disclosure.

FIG. 3 depicts an example disc surface modeling framework, according toan embodiment of the present disclosure.

FIG. 4 depicts another example method for intervertebral disc modeling,according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the principles of theinvention, reference will now be made to the embodiments illustrated inthe drawings and described in the following written specification. It isunderstood that no limitation to the scope of the invention is therebyintended. It is further understood that the present invention includesany alterations and modifications to the illustrated embodiments andincludes further applications of the principles of the invention aswould normally occur to one skilled in the art to which this inventionpertains.

The methods disclosed herein provide improvements to an automatedvertebral body segmentation method based on tandem convolutional neuralnetworks that learn the appearance of vertebral bodies and pedicles froma training set of magnetic resonance images. In one example, once thesegmentation is performed, the vertebral levels are identifiedautomatically. In another example, once the segmentation is performed,the vertebral levels are extracted from the images by an algorithm basedon connected components. In another example, intervertebral disc meshesare modeled based on the endplates of the vertebral bodies adjacent thedisc to be modeled.

Referring now to the figures, FIG. 1 is flow diagram of an examplemethod for intervertebral disc modeling, in accordance with at least oneembodiment described herein. Although the blocks in each figure areillustrated in a sequential order, the blocks my in some instances beperformed in parallel, and/or in a different order than those describedtherein. Also, the various blocks may be combined into fewer blocks,divided into additional blocks, and/or removed based upon the desiredimplementation.

As shown by block 102, the method 100 includes combining a fullyconvolutional network (FCN) with a residual neural network (ResNet). Inone example, image datasets acquired by various image scanners may leadto different data distributions from one subject to another. This maypresent a challenge when training a network. In order to overcome thischallenge and prior to designing the fully convolutional network, a biasfield correction and resampling to isotropic voxel is performed on theimage datasets. According to an exemplary embodiment, athree-dimensional anisotropic diffusion filter is applied to smoothenthe images. Further, the fully convolutional network is designed to makethe intensity distribution of all the input images similar to eachother.

As shown by block 104, the method 100 also includes training thecombined fully convolutional network with the residual neural networkfrom end to end. In one example, the residual neural network produces a512×512 output showing a predicted probability value for each pixelbased on a two-dimensional slice of size 512×512. In one embodiment, theresidual neural network contains a down-sampling and up-sampling processas shown in FIG. 2 . In this embodiment, the down-sampling process ofthe residual neural network architecture includes a convolution layerwith kernel size three, two simple blocks, and three basic blocks. Theselayers are followed by a bottleneck across layer. The up-samplingprocess is followed by a 1×1 convolutional classifier.

In one example, the network is trained using a gradient descentoptimization algorithm. For example, the network is trained usingRMSprop gradient descent optimization algorithm with decay set to 0.001.In this example, the initial learning rate is set as 0.001 and droppedto 0.0001 after two hundred epochs. In one example, a three-dimensionalfully connected conditional random field is applied on the residualneural network output to further refine the segmentation result of theresidual neural network. The designed fully convolutional network andthe residual neural network are thereby trained end to end.

As shown by block 106, the method 100 also includes receiving at leastone medical image of a spinal anatomy. In one example, the medical imageis a magnetic resonance image that has been taken of the relevantanatomy prior to a surgical procedure.

As shown by block 108, the method 100 also includes applying the fullyconvolutional network with the residual neural network to the receivedat least one medical image and segmenting at least one vertebral bodyfrom the at least one medical image of the spinal anatomy. In oneexample, given the segmented vertebral bodies, the vertebral levels arelabeled semi-automatically. For example, a user indicates the level ofthe L5 vertebra on the segmented magnetic resonance image. In thisexample, a three-dimensional connected component extraction algorithm isapplied to label different vertebral regions. In one example, theisolated components smaller than a predetermined threshold are removed.In this example, a discrete marching cube algorithm is applied on eachcomponent, followed by a mesh smoothing process using a windowedsampling function applied in the frequency domain. This is implementedas an interpolation kernel performed on each voxel. Further, dependingon what level has been defined by the user as the lower most vertebra,the remaining vertebrae are labeled in sequential order.

In one embodiment, once the three-dimensional vertebral surfaces areextracted from the segmented vertebral bodies, an intervertebral discmesh is modeled based on the endplates of the lower and superiorvertebrae. By way of example, given the inferior endplate of a topvertebra and the superior endplate of an adjacent lower vertebra, thenormals at each point of the meshes are computed, which generatesnormals for a polygon mesh. In one example, the inferior endplate of thetop vertebra is clipped by selecting the points with the normal valuesless than a predetermined threshold. In this example, the superiorendplate of the lower vertebra is clipped by selecting the points withnormal values larger than the same predetermined threshold. By way ofexample, this is done by clipping the adjacent plane, followed by smallisolated regions removed by filtering out the larger connectedcomponents of the mesh. In one scenario, the Gaussian curvature iscomputed using an anisotropic curvature filtering technique, which helpsto remove outlier curvature values from the two plates. In thisscenario, a Laplacian mesh smoothing with fifteen iterations andpredetermined relaxation factor is applied on the top vertebra and thelower vertebra to remove endplate irregularities. Further, the resultingsmoothed endplates are connected using a Poisson surface reconstructionalgorithm.

The flow diagram of FIG. 1 shows the functionality and operation of onepossible implementation of the present embodiment. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor forimplementing specific logical functions or steps in the process. Theprogram code may be stored on any type of computer readable medium, forexample, such as a storage device including a disk or hard drive. Thecomputer readable medium may include non-transitory computer-readablemedia that stores data for short periods of time, such as registermemory, processor cache, or Random Access Memory (RAM), and/orpersistent long term storage, such as read only memory (ROM), optical ormagnetic disks, or compact-disc read only memory (CD-ROM), for example.The computer readable media may be able, or include, any other volatileor non-volatile storage systems. The computer readable medium may beconsidered a computer readable storage medium, a tangible storagedevice, or other article of manufacture, for example.

Alternatively, each block in FIG. 1 may represent circuitry that iswired to perform the specific logical functions in the process. Anillustrative method, such as the one shown in FIG. 1 , may be carriedout in whole in or in part by a component or components in the cloud.However, it should be understood that the example methods may instead becarried out by other entities or combinations of entities (i.e., byother computing devices and/or combination of computer devices), withoutdeparting from the scope of the invention. For example, functions of themethod of FIG. 1 may be fully performed by a computing device (orcomponents of a computing device such as one or more processors), or maybe distributed across multiple components of the computing device,across multiple computing devices, and/or across a server.

FIG. 3 is an example illustration of a disc surface modeling frameworkaccording to an embodiment of the present disclosure. As shown in FIG. 3, image A depicts an example top vertebra model. Image B depicts anexample of how the facet normals are computed. Image C is an exampledepiction of a clipped endplate and computed Gaussian curvatures. ImageD depicts a final clipped endplate. Image E depicts a disc surfacereconstruction between the top and the lower clipped endplates.

FIG. 4 is flow diagram of an example method for intervertebral discmodeling, in accordance with at least one embodiment described herein.Although the blocks in each figure are illustrated in a sequentialorder, the blocks my in some instances be performed in parallel, and/orin a different order than those described therein. Also, the variousblocks may be combined into fewer blocks, divided into additionalblocks, and/or removed based upon the desired implementation.

As shown by block 402, the method 400 includes combining a fullyconvolutional network (FCN) with a residual neural network (ResNet).

As shown by block 404, the method 400 also includes training thecombined fully convolutional network with the residual neural networkfrom end to end.

As shown by block 406, the method 400 also includes receiving at leastone magnetic resonance image of a spinal anatomy.

As shown by block 408, the method 400 also includes receiving at leastone magnetic resonance image of a spinal anatomy applying the fullyconvolutional network with the residual neural network to the receivedat least one magnetic resonance image and segmenting at least onevertebral body from the at least one magnetic resonance image of thespinal anatomy.

As shown by block 410, the method 400 also includes extracting from thesegmented at least one magnetic resonance image (i) a firstthree-dimensional vertebral surface that corresponds to a top vertebraand (ii) a second three-dimensional vertebral surface that correspondsto a lower vertebra, wherein the lower vertebra is adjacent to the topvertebra.

As shown by block 412, the method 400 also includes modeling anintervertebral disc mesh based on a surface reconstruction that includesthe first three-dimensional vertebral surface and the secondthree-dimensional vertebral surface.

EXAMPLE 1

In one example, the performance of the approach described herein wasperformed on two publicly available datasets. The first dataset includestwenty-three subjects with T2-w turbo spin-echo three-dimensionalmagnetic resonance images of the lower spine. The images were acquiredwith a 1.5 T Siemens scanner and resampled to a voxel size of2×1.25×1.25 mm{circle around ( )}3. For each vertebral body from T11 toL5, a ground truth manual segmentation was available. In order to testthe performance of the designed automatic segmentation method on imagesfrom different scanners, a second public dataset of lower spine fromSpineWeb was also used. The second dataset includes T2-w MRI and CTscans for twenty cases with different dimensions, voxel spacing, andintensity ranges. For both datasets, manual annotations on vertebralbodies and pedicles was performed.

To increase the size of the dataset, the train datasets were augmentedusing random flipping (horizontal and vertical), sheering (with maximalrange of 0.41), rotations (with maximal range of 10), random cropping(256×256), and spline warping. The spine warping was generated usingrandom displacement vectors on a coarse 3×3 grid. The displacements weresampled from a Gaussian distribution with twenty pixels standarddeviation. Per-pixel displacements are then computed using bicubicinterpolation.

The vertebral body segmentation evaluation was compared between 2DCNN,3DCNN, and a standard ResNet, and split the dataset into 5-foldcross-validation. To create a separate validation set, 20% of the datavolumes were left out. To implement the tandem deep neural networks, theKeras and Theano libraries were used. For performance validation, themanual segmentations available on the datasets were considered as groundtruth. Further, estimations were performed on how close the results fromthe proposed method were to the ground truth annotations. Dicecoefficient, which indicates the amount of volume overlap between theautomatically segmented structures and the corresponding manuallyannotated ones were also computed. In additional, the contour meandistance (CMD) and Hausdorff distance (HD) were also calculated as theaverage and maximum distance between ground truth and automaticsegmentation, respectively.

Finally, in order to compare the MRI automatic segmentation result to CTground truth segmentations, the proposed output was registered intocorresponding CT image space on 20 patients with CT/MRI available. Insome cases, MRI scans contained more vertebral levels compared to CT,therefore the extra labels were removed from the ResNet' s ouptut.

While the inventive features described herein have been described interms of a preferred embodiment for achieving the objectives, it will beappreciated by those skilled in the art that variations may beaccomplished in view of those teachings without deviating from thespirit or scope of the invention.

What is claimed is:
 1. A non-transitory computer readable medium havinginstructions stored thereon that, when executed by one or moreprocessors, cause the one or more processors to: receive at least onemedical image of a spinal anatomy; segment at least one vertebral bodyfrom the at least one medical image of the spinal anatomy; extract oneor more three-dimensional vertebral surfaces from the segmented at leastone medical image, wherein the extracted one or more three-dimensionalvertebral surfaces comprises a first surface that corresponds to aninferior endplate of a top vertebra and a second surface thatcorresponds to a superior endplate of a lower vertebra, wherein thelower vertebra is adjacent to the top vertebra; model an intervertebraldisc mesh based on the extracted one or more three-dimensional vertebralsurfaces that correspond to the inferior endplate of the top vertebraand the superior endplate of the lower vertebra; determine normalvectors at one or more points along the first surface and the secondsurface; clip the first surface and the second surface according to apredetermined threshold that corresponds to a value associated with thenormal vectors; and model the intervertebral disc mesh based on asurface reconstruction that includes the clipped first surface and theclipped second surface.
 2. The non-transitory computer readable mediumof claim 1, wherein the instructions further cause the one or moreprocessors to: apply a down-sampling and up-sampling process to themedical image.
 3. The non-transitory computer readable medium of claim1, wherein the instructions further cause the one or more processors to:apply a trained machine learning system that includes a convolutionalneural network to process the at least one medical image.
 4. Thenon-transitory computer readable medium of claim 1, wherein theinstructions further cause the one or more processors to: apply athree-dimensional fully connected conditional random field on an outputof a residual neural network.
 5. The non-transitory computer readablemedium of claim 1, wherein the instructions further cause the one ormore processors to: receive an identification of a vertebral level onthe segmented at least one medical image; and determine identificationof one or more additional vertebral levels based on the identification.6. The non-transitory computer readable medium of claim 5, wherein todetermine identification of one or more levels includes to: apply athree-dimensional connected component extraction algorithm.
 7. Thenon-transitory computer readable medium of claim 5, wherein theinstructions further cause the one or more processors to: receive aselection of one or more determined vertebral levels; wherein to extractone or more three-dimensional vertebral surfaces from the segmented atleast one medical image is based on the received selection.
 8. Thenon-transitory computer readable medium of claim 1, wherein to segmentincludes to: apply a trained machine learning system to the received atleast one medical image.
 9. The non-transitory computer readable mediumof claim 1, wherein the instructions further cause the one or moreprocessors to: smooth the at least one medical image with athree-dimensional anisotropic diffusion filter.
 10. The non-transitorycomputer readable medium of claim 1, wherein the at least one medicalimage of the spinal anatomy includes a plurality of images; and whereinthe instructions further cause the one or more processors to: modify anintensity distribution of the plurality of images.
 11. Thenon-transitory computer readable medium of claim 1, wherein theinstructions further cause the one or more processors to: apply a biasfield correction and resampling to isotropic voxel to the at least onemedical image.
 12. The non-transitory computer readable medium of claim1, wherein the instructions further cause the one or more processors to:apply a discrete marching cube algorithm to one or more components,followed by a mesh smoothing process using a windowed sampling functionapplied in a frequency domain.
 13. A method for spinal anatomysegmentation, comprising: receiving at least one magnetic resonanceimage of a spinal anatomy; segmenting at least one vertebral body fromthe at least one magnetic resonance image of the spinal anatomy, whereinthe segmenting includes applying a trained machine learning system tothe received at least one magnetic resonance image; extracting from thesegmented at least one magnetic resonance image: a firstthree-dimensional vertebral surface that corresponds to an inferiorendplate of a top vertebra; and a second three-dimensional vertebralsurface that corresponds to a superior endplate of a lower vertebraadjacent to the top vertebra; modeling an intervertebral disc mesh basedon a surface reconstruction that includes the first three-dimensionalvertebral surface corresponding to the inferior endplate of the topvertebra and the second three-dimensional vertebral surfacecorresponding to the superior endplate of the lower vertebra; clippingthe first three-dimensional vertebral surface that corresponds to theinferior endplate of the top vertebra according to a predeterminedthreshold; clipping the second three-dimensional vertebral surface thatcorresponds to the superior endplate of the lower vertebra according tothe predetermined threshold; and modelling the intervertebral disc meshbased on a surface reconstruction that includes the clipped firstthree-dimensional vertebral surface and the clipped secondthree-dimensional vertebral surface.
 14. The method of claim 13, whereinthe method further includes: removing one or more vertebral surfaceirregularities from the interior endplate and the superior endplateprior to modeling the intervertebral disc mesh.
 15. The method of claim14, wherein the removing of the one or more vertebral surfaceirregularities includes applying a Laplacian mesh smoothing process. 16.The method of claim 14, further comprising connecting the first andsecond three- dimensional vertebral surfaces using a Poisson surfacereconstruction algorithm.
 17. A system for spinal anatomy segmentation,the system comprising: one or more processors; a computer readablemedium having instructions stored thereon that, when executed by the oneor more processors cause the one or more processors to: receive at leastone image of a spinal anatomy; apply a trained machine learning systemto the received at least one image and segment at least two vertebralbodies from the at least one image of the spinal anatomy; extract one ormore three-dimensional vertebral surfaces from the segmented at leastone image including: a first three-dimensional vertebral surface thatcorresponds to an inferior vertebral endplate of a top vertebra; and asecond three-dimensional vertebral surface that corresponds to asuperior vertebral endplate of a lower vertebra adjacent to the topvertebra; model an intervertebral disc mesh based on a surfacereconstruction that includes the first three-dimensional vertebralsurface corresponds to the inferior vertebral endplate of the topvertebra and the second three-dimensional vertebral surface correspondsto the superior vertebral endplate of the lower vertebra; clip the firstthree-dimensional vertebral surface that corresponds to the inferiorendplate of the top vertebra according to a predetermined threshold;clip the second three-dimensional vertebral surface that corresponds tothe superior endplate of the lower vertebra according to thepredetermined threshold; and model the intervertebral disc mesh based ona surface reconstruction that includes the clipped firstthree-dimensional vertebral surface and the clipped secondthree-dimensional vertebral surface.